This invention relates generally to a system and method for processing data contained in a data set and in particular to a system and method for transferring and converting data between one or more data sets having possibly different data formats and possibly requiring different special purpose conversion routines.
A computer repository for digital information, which may be interchangeable called a database, a data source or a data set, may have information about a variety of entities, such as patients in a hospital or parts in a warehouse. The information within data set may be contained in one or more data fields wherein each data field may each have its own unique data format. For example, a date data field may have a different format that a street address data field. In addition, each different data set may have data records with different data formats. However, it is often desirable to be able to export data from a particular data set into a software application as well as import data into the data set from an external data set without significant human intervention. It would take a significant amount of time to manually write a custom software application to transfer and convert data between different data sets. In addition, the integrity of the data may be jeopardized during the manual re-entry of the data because operator entry errors may be introduced into the data. Thus, it is desirable to provide a system for automatically converting data from a first data format to a second different data format.
There are a number of conventional systems and methods for transferring (e.g., importing and exporting) data between a first data set and another data set having possibly different data formats. One such system is a typical import/export software tool provided with most modern database or word processing applications. These import/export tools permit a user to import or export the data from the word processor or database into a certain predetermined number of predefined output formats, such as tab delimited data fields, SQL data records or a particular word processor format. For example, a word processing application may convert documents into a different data format that is compatible with another word processing application. These tools only permit the user to convert the data into certain predefined data formats. They do not permit a user to create a new customized data format and do not permit the data conversion routines to be customized for a particular type of data. In addition, a separate conversion routine is typically needed for each conversion instead of a customizable conversion process which may handle multiple different conversions. Some conventional tools also attempt to convert non-normalized data into normalized data which is typically used for modern database applications.
These conventional import/export tools also permit the user to convert each individual input data record into a single output data record. It is desirable to be able to generate a plurality of output data records from a single input data record or a single output record from a plurality of input data records. For example, the input data record for a merchant may be single data record containing information about the customer purchases for a day from the store, including invoice numbers, the name and address of the customer and the quantity and price of the products purchased by each customer. The store owner may wish to separate that single input data record into a plurality of output data records, such as a first data record containing information about the invoice numbers, a second data record containing information about the product purchases for inventory control, and a third data record containing information about the customer for a mailing list. Conventional import/export tools do not permit multiple output data records to be generated from a single input data record and the process of writing a custom software application to perform the conversion and transfer of data is time consuming.
It is also desirable to be able to customize the import/export tool for a particular type of data. The conventional import/export tools do not permit the tool to be customized for a particular type of data. If a data conversion to a new data format is needed, the user must hire a programmer to write a custom conversion program for the new data format. In addition, none of these conventional import/export tools have a conversion application or configuration data that may be used to customize the conversion application for a particular type of data.
Thus, there is a need for a system and method for transferring data between a first data set and a second data set having possibly different data formats which avoid these and other problems of known systems and methods, and it is to this end that the present invention is directed.